Screen content coding (SCC) is a key technology that is often applied to services, such as smart home, cloud game, medical service, and so on. The SCC technology is to capture the screen frames from all kinds of devices and to perform compression and encoding on the captured video contents having a plurality of consecutive frames. The wireless multi-screen products for sharing screen contents, such as Media Link HD, All Share Cast, etc., also define communication standard protocols for mirroring services, such as Miracast standard of Wi-Fi Display, Peer-to-Peer (P2P) wireless screen cast, to provide standard for cross-platform content sharing. Cloud interactive screen sharing service platform shares the screen frame executed on remote server to users through Internet so that users may control the remote server via the interactive interface for applications, such as cloud game, cloud smart home, remote desktop, and so on.
In the process of video coding, the inter-prediction and the intra-prediction between frames play important roles. The inter-prediction performs compression and coding based on the temporal relevance of the screen contents at different time points of the video, and the intra-prediction performs compression and coding based on the spatial relevance of the screen contents between neighboring areas in a frame. SCC often uses statistic parameters to determine whether each block of the input image is a blank block, a non-moving block or a moving block, and then selects a corresponding skip mode, intra-coding or inter-coding. Another embodiment of SCC is to classify an input image into a pre-defined class according to the characteristics of the input image, such as scene statistics, and then assigns different weight and adopts different coding parameters for each image of different class.
The contents of a screen video often include such as text, lines and graphics, which are different from usual video contents. The contents of text, lines, and graphics and the like are non-natural image and high-frequency data, and are prone to blurring or disappearance. Various coding techniques are proposed targeting screen video contents, and some coding methods are adopted by video coding standards, such as H.265/High Efficiency Video Coding (HEVC).
HEVC includes a plurality of coding modes, and motion merge mode is one of the coding modes. The motion merge mode selects at most five from the motion vectors of seven locations, either temporal (different images) or spatial (neighboring area of the same image), as candidates. The seven vectors include five motion vectors of the neighboring images and two motion vectors of a reference image. For example, for the seven coding blocks to select from, up to four may be selected from a plurality of coding blocks of the neighboring area of the same image, and up to one may be selected from the coding blocks corresponding to the same location in different images. Because some locations may not have a motion vector, at most five motion vectors are selected according to a specific order and then the motion vector with best match among the five motion vectors will be selected as the coding result. The area pointed by the motion vector is similar to the current area to be coded. As such, the compression effect is achieved by the means similar to motion compensations.
Advanced motion vector prediction (AMVP) mode is another coding mode of the plurality of coding modes in HEVC. AMVP mode compares against the five motion vectors of the neighboring image and two motion vectors of the reference image, and selects a search initial point for motion vectors. In other words, two out of seven motion vectors (the same as motion merge mode) are selected as candidates, and the motion vector with best match by comparison is selected as the coding result.
The aforementioned coding techniques or products of screen video contents use the characteristics of screen video contents and designs specific intra-coding targeting the images on the intra-prediction of text, natural graphics, mixed images of natural graphics and text. For example, HEVC SCC targets the contents of screen video on the intra-prediction of text, natural graphics, mixed images of natural graphics and text to adapt high coding complexity (such as, multiple of H.264) hardware and high memory access (such as, multiple of double data rate (DDR) memory bandwidth) to perform coding.